1. Field of the Invention
The present invention relates to an apparatus, a method, and a program which reduce fluorescent lamp flicker that occurs in a video signal that is obtained from an imaging device in a case where a subject is shot by an XY address scanning type imaging device (imager/image sensor) such as a CMOS (Complementary Metal-Oxide Semiconductor) imaging device under the lighting of a fluorescent lamp.
2. Description of the Related Art
When a subject is shot with a video camera under the lighting of a fluorescent lamp that is directly lit by a commercial AC power supply, a temporal brightness variation, i.e., so-called fluorescent lamp flicker occurs in the video signal of a shooting output due to a difference between the frequency (twice the frequency of the commercial AC power supply) of the luminance variation (light quantity variation) of the fluorescent lamp and the vertical synchronization frequency of the camera.
For example, suppose a case in which, in a region where the frequency of the commercial AC power supply is 50 Hz, a subject is shot with a CCD camera of the NTSC system (vertical synchronization frequency: 60 Hz) under the lighting of a non-inverter fluorescent lamp. In this case, as shown in FIG. 22, one field frequency is 1/60 second, whereas the period of the luminance variation of the fluorescent lamp is 1/100 second. Hence, the exposure timing at each field is shifted with respect to the luminance variation of the fluorescent lamp, so the amount of exposure at each pixel varies from field to field.
Thus, for example, if the exposure time is 1/60 second, the amount of exposure differs among time intervals a1, a2, and a3 even when the exposure time is the same, and when the exposure time is shorter than 1/60 second (but not 1/100 second as will be described later), the amount of exposure differs among time intervals b1, b2, and b3 even when the exposure time is the same.
Since the exposure timing with respect to the luminance variation of the fluorescent lamp reverts to the original timing every three fields, the brightness variation due to flicker is repeated every three fields. That is, the luminance ratio (how flicker appears) in each field varies depending on the exposure time interval, but the period of flicker does not vary.
It should be noted, however, that if the vertical synchronization frequency is 30 Hz in the case of a progressive camera such as a digital camera, the brightness variation is repeated every three frames.
Further, a fluorescent lamp typically uses a plurality of phosphors, for example, red, green, and blue phosphors to emit white light. However, these phosphors have their own unique persistence characteristics, and during the time interval from the stop of discharge to the start of the next discharge which exists within the period of luminance variation, their light emissions decay in accordance with their individual persistence characteristics. Thus, during this time interval, light appearing as white at first decays while gradually changing its hue. Thus, if the exposure timing is shifted as mentioned above, not only brightness variations but also hue variations occur. Since a fluorescent lamp has unique spectral characteristics such that a strong peak exists at a particular wavelength, variable components of the signal differ depending on the color.
So-called color flicker occurs due to such hue variations, and the differences in variable component among individual colors.
In contrast, if the exposure time is set to an integer multiple of the period ( 1/100 second) of the luminance variation of the fluorescent lamp as shown at the bottom of FIG. 22, the amount of exposure becomes constant regardless of the exposure timing and hence no flicker occurs.
In fact, a system has been conceived in which whether shooting is being done under fluorescent lamp lighting is detected by a user's operation or signal processing in a camera, and in the case of shooting under fluorescent lamp lighting, the exposure time is set to an integer multiple of 1/100 second. According to this system, occurrence of flicker can be completely prevented by a simple method.
However, since this system does not allow the exposure time to be set in an arbitrary manner, the freedom of the exposure amount adjustment means for obtaining an appropriate amount of exposure is reduced. A method is thus desired which makes it possible to reduce fluorescent lamp flicker under an arbitrary shutter speed (exposure time).
This can be realized relatively easily in the case of an imaging apparatus in which all pixels on one screen are subjected to exposure at the same exposure timing, such as a CCD imaging apparatus, because brightness variations and color variations due to flicker appear only between fields.
For example, in the case of FIG. 22, if the exposure time is not an integer multiple of 1/100 second, since flicker repeats in a period of three fields, flicker can be suppressed to a level that presents no practical problem, by predicting the current luminance and color variations from the video signal of three fields before, and adjusting the gains of the video signals of the individual fields in accordance with the prediction result so that the average value of the video signals of the individual fields becomes constant.
However, in the case of an XY address scanning type imaging device such as a CMOS imaging device, the exposure timing for each pixel is sequentially shifted by an amount corresponding to one period of the reading clock (pixel clock) in the horizontal direction of the screen, and all the pixels differ in exposure timing. Thus, flicker may not be suppressed sufficiently with the above-mentioned method.
FIG. 23 shows such a situation. As mentioned above, the exposure timing of each pixel is sequentially shifted in the horizontal direction of the screen as well. Since one horizontal period is sufficiently short in comparison to the period of the luminance variation of a fluorescent lamp, assuming that the exposure timings of pixels on the same line are the same, the exposure timings on individual lines in the vertical direction of the screen are shown. Such an assumption presents no practical problem.
As shown in FIG. 23, in an XY address scanning type imaging device, for example, a CMOS imaging device, the exposure timing differs for each line (F1 indicates this in a given field), and the amount of exposure differs in each line. Thus, brightness variations and color variations due to flicker occurs not only between fields but also within fields, which appear as a stripe pattern (the direction of the stripes themselves is the horizontal direction, and the direction of variation of the stripes is the vertical direction) on the screen.
FIG. 24 shows an on-screen flicker in a case where a subject has a uniform pattern. Since one period (one wavelength) of a stripe pattern is 1/100 second, a stripe pattern for 1.666 periods appears on one screen. Letting M represent the number of read lines per field, one period of the stripe pattern corresponds to the number of read lines L=M*60/100. It should be noted that in the specification and the drawings, an asterisk (*) is used as a symbol representing multiplication.
As shown in FIG. 25, this stripe pattern corresponds to five periods (five wavelengths) in three fields (three screens), and when viewed continuously, the stripe pattern appears to flow in the vertical direction.
FIGS. 24 and 25 show only a brightness variation due to flicker. In practice, however, the above-described color variation is also added, resulting in a considerable deterioration in image quality. A color flicker, in particular, becomes noticeable as the shutter speed becomes faster. In addition, in the case of an XY address scanning type imaging device, the influence of such color flicker appears on the screen, so the image quality deterioration becomes even more conspicuous.
In the case of such an XY address scanning type imaging device as well, if the exposure time can be set to an integer multiple of the period ( 1/100 second) of the luminance variation of a fluorescent lamp, the amount of exposure becomes constant regardless of the exposure timing, so fluorescent lamp flicker including on-screen flicker does not occur.
However, if the electronic shutter speed is made variable in a CMOS imaging device or the like, the imaging apparatus becomes complex. Further, even in the case of an imaging apparatus whose electronic shutter can be shut off freely, if the exposure time can be set to only an integer multiple of 1/100 second to prevent flicker, the freedom of the exposure amount adjusting means for achieving an appropriate exposure is reduced.
Accordingly, there have been proposed various methods for reducing fluorescent lamp flicker unique to an XY address scanning type imaging device such as a CMOS imaging device. For example, Japanese Unexamined Patent Application Publication No. 2004-222228 discloses an invention in which fluorescent lamp flicker unique to an XY address scanning type imaging device such as a CMOS imaging device can be detected with high accuracy and reliably and sufficiently reduced solely by simple signal processing without performing complicated processing such as detecting a flicker component by using a light receiving device, regardless of a subject or the level of a video signal, the type of a fluorescent lamp, and the like.